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A study designed to examine several student and instructor characteristics to 
see what relationship each had with student ratings of their instructors concentrated 
on such variables as (1) student sex. age, ma|or. level of education, grade point 
average, or course grades previously received from the instructor m relation to 
student ratings of teachers, and (2) instructor sex, age. faculty rank, degree held, 
major area, or length of teaching experience with respect to ratings by students.. A 
2-page student scale provided for ratings of instructors on each of three 7-point 
continuums: (1) aloof, egocentric, restricted behavior versus friendly, understanding 
behavior, (2) evading unplanned, slipshod behavior versus responsible, systematic, 
businesslike behavior, and (3) duk routine behavior versus stimulating, imaginative, 
enthusiastic behavior. Student scales were administered to 4.402 students m dasses 
taught by 87 different instructors. Simultaneously an instructor characteristics scale 
was administered in each of the classes. The study showed that student ratings are 
not influenced even by variables such as grades previously earned from the instructor 
being rated, that ratings were different among various departments within the School 
of education, and that younger instructors received higher ratings than older 
instructors in each category. (DG) v 
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Introduction 



Mr. Fred M. Hechinger, education editor of the New York Times has 
written; 

^Students have persuaded the educational leadership 
that the student's voice on campus matters deserves 
to be heard and that their complaints about inadeauate 
teaching and a curriculum out of step with contemporary 
society are often justified” (Hechinger, 1965). 

He further states that students now seek to sit on educational policy- 

making committees, a responsibility that faculty members often shirk. 

These student complaints are dramatized by unrest on 
throughout the country. Students at large universities complain about 
being disturbed and somewhat threatened by the dehumanization and the 
loss of identity they perceive in the multiversity. Many are deeply 
concerned about the nature of our system of higher education and want 
change, most noteworthy in the improvement of instruction. 

Frederick Rudolf, professor of history at Williams College, finds 
nothing new in the desire of students to bring about change and reform 
on American campuses. He states: "For a few: years the college is the 

oyster and they (students) will have it served up exactly as they wish 
it, unless there are those who help them to some other, perhaps wiser 
choice (Hechinger, 1965)." Some faculty ai,d administrators are helping 
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by cooperating with student organizations wanting to improve instruc- 
tion. At the City College of New York, a Joint student-faculty com- 
mittee was established to develop in a truly collaborative fashion some 
type of course evaluation, A program evolved consisting of the distri- 
bution of course-instructor questionnaires to instructors on a volun- 
tary basis. The instructor distributed the scales, collected them 
and was free to use them in any way he wished. He could send them to 
the office of the Chairman of the Department of Student Services where 
a confidential report would be prepared and returned to him. Of 400 
instructors requesting to participate, only 144 sent the collected 
data to the Department of Student Services for scoring. 

Other faculty groups are responding to students who want to have a 
greater role in the affairs of their institution. In 1965, Cornell 
University appointed a Committee on the Quality of Undergraduate In- 
struction chaired by Dr. Alfred E. Kahn (1965). Although appointed by 
the executive branch of the university, the membeis worked closely with 
the student body to study the academic aspects of undergraduate educa- 
tion at Cornell. The comprehensive report that resulted from the study 
included the following findings: 

1. Grossly inadequate teaching occurs in more in- 
stances than is tolerable, 

2. There can be no doubt that student dissatis- 

faction with undergraduate instruction has 
basis in fact. " 

3. There is only one ultimate determinant of the 
quality of undergraduate instruction and only 
one ultimate source of its improvement - the 
individual teacher himself. 

The committee recommended unanimously that ” teachers obtain anony- 
mous student evaluations of their courses'* and, with one member dis- 
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senting, “that the university give Student Government financial and 
technical assistance in preparing and publishing competent and respon- 
sible campus-wide student evaluations of courses.” 

A similar committee at Michigan State University, The Committee on 
Undergraduate Education, was appointed in 1967 by the school’s president 
to reexamine the undergraduate program and to propose desirable changes. 
In a section of their final report concerning student evaluation of 
teaching, the committee stated that ”an instructor should know when 
students do not approve of a particular technique or approach (Adams, 
1967). The committee recommend that a special all-university office 
by given the responsibility and adequate support: to revise or develop 

new Student Instructional Rating Forms; to collect and disseminate 
normative data relative to these forms; and to analyze the results and 
prepare summary reports.” They further recommended that ’’Faculty mem- 
bers be encouraged to have these summary reports sent to their depart- . 

ment chairman so that the results can be involved in decisions per- 
taining to them.” 

The number of colleges and universities using some method of. stu- 
dent evaluation is increasing rapidly and not without some adverse 
consequences. Obviously, there are many dangers when student rating 
of faculty performance may be a factor in a teacher's professional 
standing, possibly affecting his promotion. 

if 

If studies are not methodologically sound, popularity might rate 
higher than intellectual rigor; or political views might influende 
decisions that ought to be arrived at by a process of free inquiry. 

Other questions concerning the validity of student ratings are 
equally critical, and this report summarizes a study designed to 
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examine several student and instructor characteristics to see what 
relationship each had with student ratingsof their instructors. 

The research was carried out within the School of Education at 



Colorado State College (Greeley, Colo,) during the 1967 winter quarter. 
The spec'xfic instructor population was identified as all faculty with- 
in the School of Education who were teaching courses during winter 

quarter. The student population was all those who were enrolled in 
those courses. 



No attempt was made to identify effective and ineffective in— 
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structors. Instead, the study concentrated upon variables believed 
to have some relationship to student ratings. Therefore, the major 
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questions studied were: ^ 

1# Does student sex, age, major, level of education, grade point 
average, or course grades previously received from the instruc- 
tor being rated have any relationship to ratings of instructors? ^ 

2. Are instructors who differ in sex, age, faculty rank^ degree 
held, major area, or length of teaching experience rated 
differently by students? 

# 

A further question concerned the accuracy of students’ reports of 
their grade point averages (gpa). This was done to check the validity | 
of student information. If large discrepancies existed l>etween stu- 

; 

dents' reported gpa and their actual gpa (determi/sd from college 
records) the remaining information provided by students would be suspect. 

To collect the necessary data, a one-page Student Scale was con- ? 

structed (see attached) . The scale asked students to provide infer- I 

mation about themselves and to rate their instructor on each of three 
seven-point continuums. The referents on each continuum were identi- 
fied by Ryans as being important teacher traits or patterns of behav- 
ior (Ryans, 1960). These were: 






I. 


aloof, egocentric, 
restricted behavior 


vs. 


friendly, understanding, 
behavior 


II. 


evading, unplanned, 
slipshod behavior 


vs. 


responsible, 

businesslike 


systematic , 
behavior 


III. 


dull, routine 
behavior 


vs. 


stimulating, 

enthusiastic 


imaginative 

behavior 



A short Instructor Characteristics Scale was designed to obtain 
from faculty such information as sex, age, rank, degree held, major 
area and length of teaching experience. 

Next, a group of doctoral students were trained to administer the 
scales. At the instructors* convenience, 4,402 Student Scales were 
administered in the existing classes taught by 87 different instruc- 
tors. At the same time, the Instructor Characteristic Scale was 
administered. 

Correlation analyses were performed to answer the questions. One 
statistical technique used to analyze the data was multiple regression 
as presented by Ward and Bottenberg (1963). In this approach the con- 
tribution of a variable or a set of vax*iables to estimation of a cri- 
terion was measured by the difference between two multiple correlation 
coefficients (RSQ*s); one obtained for a regression model in which 
all the predictors were used, and the other (RSQ) obtained for a re- 
gression model in which the variable or subset of variables under 
examination was deleted (Ward, 1962), 

I 

FINDINGS 

Based on statistical analysis of data collected on 87 instruc- 
tors from 4,285 usable Student Scales, the following conclusions were 
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reached: 

1. Students remembered and accurately reported their grade point 
averages. The correlation for lOD randomly selected student- 
reported gpa»s and their actual gpa’s was .96. 

2. Student ratings of instructors were not substantially related 
to student’s sex, age, grade level, major area, grade point 
average or grade (s) previously received from the instructor 
they "rated (Table I). 



TABLE I 

CORRELATIONS AND RSQ*s BETWEEN CRITERIA AND STUDENT CHARACTERISTICS 



Student Characteristic (s) 



Rating Continuum 


Reported GPA 


Previous Grade (s) 


(sex. Age, 
Major) 


Level , 




N 


r 




r 




Tl5g 


I 


4,032 


-.03 


753 


.14 


4,285 


.006 


II 


4,032 


-.02 


753 


-.01 


4,285 


.015 


III 


4,032 


.03 


753 


.18 


4,285 


.019 



As indicated in Table I, the highest correlation (.18) was 
obtained between students previous grade (s) earned from the 
instructor they were rating and their rating of that same 
instructor on continuum III. Also, there was no relation- 
ship, reflected by correlations of -.03, -.02 and .03, be- 
tween students grade point average and their ratings. 

When the student characteristics of sex, age, grade level, 
and major were used collectively to predict ratings, the 
RSQ*s were all less than .02. To demonstrate the low pre- 
dictability of this set of characteristics, the regression 
weights associated with each characteristic were used to 
predict how two hypothetical students would have rated an 
instructor. These predictions are found in Table II. 




TABLE II 



PREDICTED INSTRUCTOR RATINGS FROM STUDENT CHARACTERISTICS FOR TWO 

HYPOTHETICAL UNLIKE STUDENTS 



— - . ^ predictea Rat- 

Characteristics of Student ing on 

Continuum 



Student 


Sex 


~ Age ' 


Level of Educ. 


Major 


1 


11 


ill 


1 


Male 


35 


Graduate 


Educ. Admin 


3.5 


2.7 


2.7 


2 


Female 


18 


Freshman 


Elem. Educ. 


3.3 


2.8 


2.3 








* 


Difference 


0.2 


0.1 


0.4 



Based on the regression equation, in which sex, age, level 
of education > and major area were used. Student No. 
35-year-oId male graduate student majoring in educational 
administration would be predicted to rate an instructor 
3.5 on continuum I, 2.7 on continuum II, and 2.7 on con- 
tinuum III. Another student (Mo. 2), an 18-year -old female 
majoring in elementary education was predicted to rate an 
instructor 3.3, 2.8, and 2.3 on the three respective con- 
tinuums. These small differences in predicted ratings 
(0.2, 0.1, and 0.4) for quite dissimilar students dv^mon- 
strated the low relationship between this group of student 
characteristics and student ratings of instructors. 

3. Generally, mean student ratings of instructors showed more 
relationship to instructor characteristics (Table III) . 



Insert Table III 



For the instructor characteristics of sex, age, degree held, 
faculty rank, and number of years experience, collectively, 
the RSQ was .27 for continuum II (**evading vs. businesslike'’). 
This means that 27% of the variability of ratings on con- 
tinuum II is related to this set of instructor characteris- 
tics. 

4. The greatest reduction in the index of predictability (RSQ) 
was observed when the "department” classification variable 
was deleted from the equation. The RSQ value for continuum I 
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TABLE III 

RSQ*s FOR INSTRUCTOR CHARACTERISTICS AS PREDICTORS OF STUDENT 
MEAN RATINGS ON CRITERIA FOR EIGHTY-SEVEN INSTRUCTORS 







Variables 


Continuum 


RSQ 


Model 


Instructor Variables 


Deleted 


Predicted 


Value 


1 


Sex, Age, Rank, Degree, 


None 


I 


.15 




Dept. , Yrs. Exp. 




II 


.27 








III 


.18 


2 


Age, Rank, Degree, Dept., 


Sex 


I 


.15 




Yrs. Exp. 




II 


.27 








III 


.18 


3 


Sex, Rank, Degree, Dept., 


Age 


I 


.15 




Yrs. Exp. 




II 


.19 








III 


.18 


4 


Sex, Age, Degree, Dept., 


Rank 


I 


.15 




Yrs. Exp. 




II 


.24 








III 


.18 


5 


Sex, Age, Rank, Dept., 


Degree 


I 


.15 




Yrs. Exp. 




II 


.27 






* 


III 


.18 


6 


Sex, Age, Rank, Degree, 


Dept. 


I 


.05 




Yrs. Exp. 




II 


.14 








III 


.08 


7 


Sex, Age, Rank, Degree, 


Yrs. Teach- 


I 


.14 




Dept. 


ing Exp. 


II 


.25 








III 


.16 


8 


Sex, Dept. 


Age, Rank, 


I 


.11 






Degree, Yrs. 


II 


.13 






Exp. 


III 


as 







0 
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( 1 ) 

( 2 ) 

(3) 

(4) 

(5) 

( 6 ) 

(7) 



dropped from .15 to .05; for continuum II from .27 to .14- 
and for continuum III from .18 to .08. ’ 



Mean student ratings according to major areas reflected this 
reduction in RSQ's. For example, means calculated for each 
department on the »*aloof \>^s. friendly^’ continuum ranged from 
4.0 to 5.0, on the ’’unplanned vs. systematic” scale from 3.9 

to 5.4, and on the ’’dull vs. stimulating” scale from 4.1 to 

5 # 2 # 



5. 



The correlations of the three student rating criteria with 
instructor age, faculty rank, highest degree held, and years 

©xperience yielded twelve correlations varvin&r 
from -.27 to +.15 (Table IV). ^ ^ 



TABLE IV 



INTERCORRELATION COEFFICIENTS BET^VEEN INSTRUCTOR 
CHARACTERISTICS AND THEIR AVERAGE RATING BY STUDENTS 



Variable 



Continuum I 
Continuum II 
Continuum III 
Age 

Faculty Rank 
Highest Degree Held 
Yrs. Teaching Experience 



1 


2 


3 


4 










' .47 


.71 


.67 


' 




-.17 


-.15 


-.25~ 




-.15 


.15 


-.06 


.51 


-.16 


.12 


-.06 


.37 


-.18 


.00 


-.27 


.86 











.79 

.54 



37 



Nine of these were negative, suggesting that older, more edu- 
cated and more experienced instructors were rated lower than 
younger and less experienced instructors. Further, because 
student ratings were used as an indicator of each stu- 
dent’s rating, correlations were attenuated. If each student’s 
rating scores were used, correlations would have been 
higher and the inverse relationship magnified. 



DISCUSSION 

To involve students in any system of faculty evaluation seems per; 
ilous. Yet it is students who have, the most contact with instructors. 
To ignore them seems even more perilous. The results of this study 
confirmed that student ratings are not influenced even by variables 
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such as grades previously earned from the instructor being rated. 

Since student ratings were found to be relatively unbiased, more confi- 
dence could be placed on the results of how certain instructor charac- 
teristics were reflected in students* ratings. As was mentioned, these 
results suggested that ratings were different from various departments 
within the School of Education. 

Further, the younger instructors with less faculty rank, and fewer 
years of teaching experience were consistently rated more "friendly, 
understanding; systematic, responsible, businesslike; stimulating, 
imaginative, and enthusiastic.*' 

Probably as important as the specific findings, the investigator 
made several observations that did not lend to statistical analysis: 

a. The multiple regression approach used in this study was 
effective in analyzing the degree to which certain student 
and instructor characteristics were reflected in student 
ratings. 

b. Ryans(1960), using factor analysis, identified three patterns 
of teacher behavior in the Characteristics of Teachers study. 
However, in the present study, when each of these patterns 
was placed on a rating continuum and administered to college 
students, the inter -correlations among ratings were high 
(Table IV) . This suggested that the traits Ryans identified 
as being independent do overlap. Therefore, a total student 
rating score of an instructor could have been obtained by 
adding the student ratings made on each continuum. 

c. The study gave a large portion of the student body a chance 
to rate their instructors. Often, when such a route is not 
provided or such a desire is thwarted, the upshot may be a 
poorly organized student-directed evaluation resulting in a 
faculty "blurb" sheet. 

d. After student leaders in charge of developing an instructor 
evaluation program talked with this investigator, they 
appreciated the time, sophistication, a^d financing needed to 
carry out a methodologically sound student evaluation. And, 
after examining the complexity of the issues, these student 
leaders agreed that some degree of confidentiality should 
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be maintained and that student publications on course* evalua- 
tions should contain more objective information such as* the 
number and type of exams given.. 

SUMMARY 

As a consequence of the study reported here, we now have more in- 
formation on the nature of college students ratings of their instruc- 
tors, at least at Colorado State College. Vie have shown that some 
information reported by students is accurate. We have shown that stu- 
dent ratings are not influenced by such variables as previous grades 
received from the instructor they rated. And we have shown that instruc- 
tors who differ on certain characteristics are rated differently by 
their students. 

Sfill other equally important issues remain unanswered. We need 
to know what student characteristics, if any, do influence their rating 
of instructors. We need to know if a parallel study performed within 
another academic area or at another institution will yield similiar 
results. And, we need to know if there are other i«is true tor character- 
istics that better reflect student ratings. 

Yet, in terms of students rating their instructors, the question: 

•*to rate or not to rate*’ can best be answered by citing a quotation 
attributed to Aristotle; "You get a better notion of the merits of the 
dinner from the dinner guests than you do from the cook.” 
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STUDENT SCALE A Cod t C Codt 

B Code D code 



The School of Bduettion at Colorado State College la involved in a reaeereli etti^ to examine 
the nature of student ratingt of inatruetora. We ere asking you to participate by providing 
the Informetion found on this scale* You can be asaured that the information you provide 
vlll be considered confidential -and will be coded to insure your anoiqnslty* To insure the 
validity of the study it is extremely important that you complete the following information 
to the best of your ability* 

1. NAME (please print ) 

last first 

IX. 1N3TBDCT0R RATING 

Please indicate by auking one mark on each of the three horisontal lines » where you feel^ 
the instructor in this course is located: 



aloof “^1 
egocentric i 
restricted | 
^ behavior 1 | 






I 


-L ^ 


1 


.1 


F friendly “1 
1 underatandiiig I 
1 1 behavior 




1 


i 


3 




5 


4 





evading 


1 














I 


F systematic 


slipshod 


















responsible 


unplanned 


L 






1 


_j 








businesslike 


^ behavior 1 




_i 


1 


_j 


-j ! 


1 behavior ^ 



r 2 3 4 5 A 



F dull 
I routine 
t behavior^ 


i 




1 


-J- - 


j 


i 


i 


J 


F stisailatiag 
imaginative 
enthusiastic 
J>elia»ior^ 






1 


2 


3 


4 


5 


6 







III. STUDENT CHARACTERISTICS 

Please provide the following information about yourself to the best of your ability: 



A, Sex: HsleCO ) f emsled ) 

B* YOur age (in years to last birthday); 

C. Cheek level of Education: 9< >Fros« 2( )Soph. 3( )Jr. 4( )Sr. 5( )6rad. B( )Other J 

D* Check the category below that describes your area of study: 

l( )Blem. Bduc* 4( )Educ. Admin. 7< >Mnst. & Research 

2( )8ec« Bduc. 5( )fsych* Coun. & Cuid. S( )Other 

3( )Spee. Bduc. 6( )C.S.P.V. 



B. a. If you have previously taken one course from the instructor you are rating, what number 
grade did you receivef (nua^r should be in terms of a Appoint system) Hum. trad e 

b* If you have previously taken more than one course from this instructor, what is the 
average of those gradesT (Report to one decimal place on 4*point system) . 

F. What is your overall cumulative Grade Point Average to date? 

(Report to one decimal place on a 4-point system) Overall GPA - •- - 



